Direct Text to Speech Translation System Using Acoustic Units

This paper proposes a direct text to speech translation system using discrete acoustic units. framework employs in different source languages as input generate the target language without need for transcriptions this language. Motivated by success of units previous works systems, we use same pipeline extract encoder combined with clustering algorithm. Once are obtained, an encoder-decoder architecture is trained predict them. Then vocoder generates from Our approach was tested on new CVSS corpus two mBART models employed initialisation. The systems presented report competitive performance most pairs evaluated. Besides, results show remarkable improvement when initialising our proposed model pre-trained more languages.